A Hierarchical Design Scheme for Application of Augmented Reality in a Telerobotic Stereo-Vision System

Operating a robot from a distant place using haptic and visual feedback gets enormous applications in various situations. But due to time-delay in the communication, an tele-operator sometimes has to go for a move-n-wait strategy. However, the problem can be minimized by using Augmented Reality (AR) concepts of superimposing virtual objects onto the real video image of the remote scene to create a simulation plan in the local machine. Operator can make trial and error to finalize his plan. This increases task safety and also reduces realtime network interaction by transferring only the finalized trajectory data. In this paper we have presented a hierarchical design scheme for developing such an AR system. At first a geometric model of a six Degree of Freedom(DOF) robot arm is developed. Based on that model state-of-art graphics system is used to model a 3D graphical arm. Then the graphics is superimposed onto the real image using accurate camera calibration and registration methods. Algorithms are also developed for activating motion in the visualization system. A graphical user interface is designed to facilitate the task simulation. The design scheme is implemented and tested using Microsoft .NET framework, visual C#.NET and Microsoft DirectX with a stereovision system comprising of a PUMA-560 robot and operating over a LAN.